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PREFACE 

This report describes part of a comprehensive and continuing pro- 
gram of research concerned with advancing the state-of-the-art in re- 
mote sensing of the environment from aircraft and satellites. The 
research is being carried out for NASA T s Lyndon B. Johnson Space Center, 
Houston, Texas, by the Environmental Research Institute of Michigan 
(ERIM). The basic objective of this multidisciplinary program is to 
develop remote sensing as a practical tool to provide the planner and 
decision-maker with extensive information quickly and economically. 

Timely information obtained by remote sensing can be important to 
such people as the farmer, the city planner, the conservationist, and 
others concerned with problems such as crop yield and disease, urban 
land studies and development, x*ater pollution, and forest management. 

The scope of our program includes: 

1. extending the understanding of basic processes 

2. discovering nex^ applications, developing advanced remote- 
sensing systems, and improving automatic data processing 
to extract information in a useful form 

3. assisting in data collection, processing, analysis, and 
ground-truth verification. 

The research described herein xras performed under NASA Contract 
No. NAS 9-149 88, Task 5. R. E. Joosten (TF5) was the NASA Task Tech- 
nical Monitor. The program was directed by Richard R. Legault, Vice 
President of ERIM and Head of the Infrared and Optics Division, Quentin 
Holmes, Head of the Information Systems and Analysis Department, and 
Richard F. Nalepka, Principal Investigator and Head of the Multispectral 
Analysis Section. William A. Malila served as Task Leader. 

The authors xtfish to acknoxjledge the technical direction and assist- 
ance provided by R. F. Nalepka and W. A. Malila. The ERIM number of 
this technical memorandum is 122700-4-R. 
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ADDITIONAL STUDIES OF FOREST CLASSIFICATION ACCURACY AS INFLUENCED 

BY MSS SPATIAL RESOLUTION 


SUMMARY 

This technical memorandum presents the 
results of Investigations conducted in order 
to address two unresolved issues raised in 
the report entitled, "Forest Classification Accur- 
acy as Influenced by Multispectral Scanner Spatial 
Resolution" [1] . 

These issues were: (1) the large variation 

in classification accuracy manifested by individual 
forest features for any one case of spatial resolu- 
tion and the inconsistent changes in classification 
accuracy demonstrated among features as spatial re- 
solution was degraded, and (2) the large proportion, 
of unclassified resolution elements that appeared 
in coarse resolution data as a result of using a 
constant classification rejection threshold- The 
respective approaches of these investigations con- 
sisted of a -..^nature analysis to help explain class- 
ification performance for individual features and a 
variation of the rejection threshold for classify- 
ing coarse resolution data. 

For the signature analysis, two-channel 
ellipse plots for pairs of chanrr Is most optimal 
for discriminating the features were generated to 
illustrate the more significant statistical differ- 
ences and similarities among the multivariate signa- 
ture distributions. Analysis of the plots showed 
that the capability of signatures to identify their 
respective features is dependent on the amount of 
statistical overlap among signatures which causes 
resolutif i' element mlselassif ication. Reductions in 
signature variance that occur in data of degraded 
spatial resolution may not necessarily decrease the 
amount of statistical overlap among signatures having 
large variances and small mean separations. Fea- 
tures classified by such signatures may thus con- 
tinue to have similar amounts of mlselassif led ele- 
ments in coarser resolution data, and thus, not neces- 
sarily improve In classification accuracy. 

PAGE IS 1 
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A simulation of feature classification 
was performed under the assumption that the data 
values within feature training areas had multi- 
variate normal distributions at each ease of spa- 
tial resolution. Comparison of the simulated to 
the real classification results suggested that the 
real data values within training areas tended to 
approach a normal distribution more closely as 
spatial resolution was degraded. Additional anal- 
ysis showed that small differences in rates of de- 
crease among signature determinant values existed 
as .spatial resolution was degraded * Finally, evi- 
dence was provided to suggest that correlations 
between channels of a spectral signature decrease 
as spatial resolution is degraded. 

Adjusting the classification rejection 
threshold to maintain a constant proportion of 
unclassified resolution elements among all cases 
of spatial resolution caused significant increase ■ 
in the overall classification accuracy of (64 meters) 
data. Because unclassified elements are typically 
considered to be classification errors, results of 
the previous study [1] had shown that (32 meters )^ 
resolution provided the most accurate element-by- 
element classification of the data, due to a 
large proportion of unclassified elements in 
(64 meters) 2 data. If the proportion of unclassi- 
fied elements is reduced by adjusting the rejection 
threshold, then (64 meters)^ resolution provides the 
best overall classification performance. The recom- 
mendation for future operational systems is 
that rejection thresholds be selected with great 
care, and not set at an arbitrary constant value. 
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1 

INTRODUCTION 

The purpose of this technical memorandum is to provide the 
results of investigations into the influence of multispectral scanner 
spatial resolution on forest classification accuracy. These Investi- 
gations were conducted in order to address two unresolved issues raised 
in a recent report entitled "Forest Classification Accuracy as Influenced 
by Mul it spectral Scanner Spatial Resolution" [1], Specifically, 
these issues were: a) the large variation in classification accuracy 

manifested by individual condition class forest features and the incon- 
sistent changes in classification -ccuracy demonstrated among features 
as spatial resolution was degraded; and b) the large proportion of 
unclassified resolution elements that appeared in coarse resolution data 
as a result of using a constant classification rejection threshold* 

1.1 RESULTS OF PREVIOUS STUDY 

For the study cited, the influence of multispectral scanner (MSS) 
spatial resolution on forest classification accuracy was determined for 
forest features at several levels of detail ox hierarchies that might be 
appropriate for detailed in— place inventories and nationwide forest 
surveys. The most detailed hierarchy of features constituted condition 
classes (forest cover types differentiated Into stands on the basis of 
age and size class) that were identified on U.S. Forest Service maps. 

A supervised classification procedure that utilized signatures extracted 
from training areas in each such feature was used to classify MSS data 
of inherent (2 meters) 2 spatial resolution collected over two separate 
ground areas of the Sam Houston National Forest, Classification perfor- 
mance, determined, for the hierarchy of condition class features, was 
stated for three regions within the data set that included the training 
areas within each feature area, entire feature areas with boundary elements 
excluded (boundary exclusive test sets), and entire feature areas with 

3 
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boundary elements included (boundary inclusive test sets) . The results 
were then aggregated to provide a measure of classif icatron performance 
for more general hierarchies of features. For the five pine features 
within Data Segment 1, condition classes were combined into cover type 
features on the basis of species, and alternately, into teatures based 
on maturity that were called growth stages. For the most general hier- 
archy 3 all pine sawtimber features were combined into single physiog- 
nomic feature for Data Segment 1 and two condition classes of hardwood 
sawtimber were combined into a single physiognomic feature within Data 
Segment 2. 

The data were then progressively coarsened to simulate five addi- 

2 2 2 2 2 

tional data sets having (4) , (8) , (16) , (32) and (64 meters) 
spatial resolution. Similar processing and analysis of all spatial 
resolutions determined that classification accuracy improved for all 

2 

hierarchies of features as spatial resolution degraded from (2 meters) 
to (32 meters) ^ when conventional single-element multispectral processing 
procedures were used. Improvement in classification accuracy as resolu- 
tion degraded was attributed to a reduction in the number of misclassif ied 
resolution elements that occurred as a result of reduced competition 
among signature distributions. Reduced competition presumably resulted 
from a reduction in scene variation that is inherent in the averaging 
of information over larger ground areas. Improvements in classification 
accuracy noted for hierarchies of more general (aggregated) forest features 
occurred by virtue of the fact that misclassifications of resolution ele- 
ments be tween certain specific features cancelled for their aggregated 
feature class. 

Two somewhat unexpected phenomena were noted as part at the results . 
One concerned the classification performance for individual forest 
features. In determining the general significance of spatial resolution, 
overall classification accuracies for hierarchies were derived from the 
total correct classification for all features contained within the 
hierarches and found to improve as resolution degraded. However , results 

4 
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for individual features within the hierarchy of condition classes showed 
that classification accuracies can vary widely from feature to feature 
for any one case of spatial resolution. More importantly, the trends 
in classification accuracy as a function of spatial resolution were not 
entirely uniform from feature to feature. Some features were consist- 
ently better classified in coarser resolution data and some were not. 

A second unexpected phenomenon resulted from the use of a constant 

classification rejection threshold fcr all cases of spatial resolution. 

2 

When spatial resolution was degraded to (64 meters) , the percentage 
of unclassified elements in test sets increased greatly. Because 
unclassified elements were considered to be errors, this caused a net 
decrease in classification accuracy. -'7 

1.2 ISSUES ADDRESSED IN THIS MEMORANDUM 

Reported herein are the results of two investigations whose 
respective appreaches consisted of a signature analysis to help explain 
classification performance for individual features and a variation of the 
rejection threshold for classifying coarse resolution data. 
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FEATURE SIGNATURE ANALYSIS 

2.1 BACKGROUND 

Figures 1 and 2 display the classification accuracies that were 
achieved previously [1] at each case, of spatial resolution for indi- 
vidual features in each of the two data segments. These accuracies 
were achieved for training areas from which the signatures were 
extracted. ..Classification accuracies for whole feature areas were 
shown in Ref. 1 to be lower, due to the larger variation across the 
entire area of a feature that may not be represented within the 
training area. By showing classification performance for training 
areas, we represent an upper limit of performance that assumes each 
feature area is adequately described by its respective signature(s) . 
Thus, the results are uncomplicated by the additional confusion to 
the classification performance that may be introduced by nonuniformi- 
ties within non-training portions of the features, boundary elements, 
etc. 

Both figures illustrate wide variations in feature classification 

accuracy for any one case of spatial resolution. For example, with 
2 

(2 meters) data in Data Segment 1, 51.1% of the Pine Regeneration 
Feature was classified correctly while Mature Loblolly Pine was only 
16.9% correct. The discrepancies among feature classification accu- 
racies are even greater for Data Segment 2, with a 52.9% difference 

in accuracy between the two hardwood condition class features in 
2 

(2 meters) data. 

: For Data Segment 1, three of the features display more or less 

consistently better classification accuracies as spatial resolution 

degrades, while the remaining two generally stay low, undulating 

slightly from better to poorer performance and back, out to the 
2 

(32 meters) case of resolution. The dramatic increases m accuracy 
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Spatial Resolution (meters) - 


FIGURE 1. CLASSIFICATION ACCURACIES FOR INDIVIDUAL CONDITION CLASS FEA- 
TURES WITHIN DATA SEGMENT 1 AT EACH CASE OF SPATIAL RESOLUTION 
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for (64 meters) ^ data are probably due to the fact that one less sig- 
nature was used to classify training areas because one class contained 
insufficient resolution elements to form a valid signature. 

For Data Segment 2, classification accuracies for all features 

2 

generally improve out to (32 meters) . [Immature Loblolly Pine pro- 

2 

vides the one exception for (4 meters) resolution.] Decreases in 
accuracy for the Cutover Feature and Immature Laurel Oak/Willow Oak 
Sawtimber at (64 meters) 4 " were caused by a substantial increase in the 
number of resolution elements that were unclassified (See Sec. 3). 

2.2 PROCEDURE 

To analyze feature signatures, we generated two-channel ellipse 
plots of the signatures in each data segment for several cases of 
spatial resolution. For each case of spatial resolution! the plots 
of signature ellipses provided a graphic illustration of the relative 
locations, sizes, and orientations of the feature signatures in two- 
channel space. By choosing pairs of channels most optimal for the 
discrimination of all features, such two— channel plots can illustrate 
the more significant statistical differences and similarities among 
the multivariate signature distributions. It was found that much of 
the classification performance for the forest features could be 
explained by noting the relationships among signatures in each plot. 

To gain further insight into the effect of degraded spatial 
resolution on the feature signatures, we simulated classification per- 
formances under the assumption that the data values within feature 
training areas at each case of spatial resolution had multivariate 
normal distributions , and also examined relative rates of decrease of 
signature determinants and dispersion volumes. Departures from nor- 
mality on the part of the actual data distributions and non-uniform 
rates of change in dispersion volume among the signatures may offer 
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additional explanations of the observed trends in classification 
accuracies for individual features . 

To determine the departures from normality for the actual data 
distributions within the training areas, we compared the ideal per- 
formance to be expected for classifying data values known to be dis- 
tributed in a normal manner to the classification performance achieved 
for the real data. The ideal expected classification performance for 
each feature was simulated by computing classification probabilities 
among the signatures of each data segment. For each signature taken 
in turn, 2000 data values were generated at random according to the 
signature's specified multivariate normal distribution and then classi- 
fied appropriately as one of the feature signatures. The fractions 
of data values assigned to each feature signature provided an estimate 
of the probability that data values within one feature area would be 
correctly classified, misclassif ied as another feature, or remain 
unclassified. The classification algorithm, decision boundaries, and 
classification rejection threshold were the same as those used to 
classify the real data values in the training areas. 

To examine the relative rates of decrease for signature determi- 
nants, we plotted the determinant values of all signatures in Data 
Segment 1 as a function of spatial resolution. A final plot, showing 
the ratio of dispersion volume to the product of the 11 channel 
standard deviations for two signatures as a function of spatial 
resolution, was made to allow observation of the relative amount of 
between-channel correlations of the signature distributions as resolu- 
tion was degraded. 

2 . 3 RESULTS 

2.3.1 ANALYSIS OF SIGNATURE PLOTS 

Figures 3-6 illustrate, for two dimensions, the signature dis- 

2 

tributions for features of Data Segment 1 at resolutions of (2) , 

2 2 2 

(8) , (32; , and (64 meters) , respectively. Figures 7 and 8 
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FIGURE 6. SIGNATURE DISTRIBUTIONS FOR CONDITION CLASS FEATURES OF DATA 
SEGMENT 1 (64 METERS) 2 RESOLUTION 
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illustrate feature signatures of Data Segment 2 at (2) and (64 meters) . 
A test of spectral channel performance for feature discrimination indi- 
cated that the pairs of channels illustrated for each data segment 
proved to be among the best for separating the respective features at 
all cases of spatial resolution. However , the use of all 11 channels 
when classifying the data values offered considerable improvement for 
feature discrimination. 

At (2 meters) , the largely overlapping signature distributions 
in each data segment obviously offer the least likelihood for success- 
ful discrimination of features. The large variance for each signature 
provides evidence of the spectral irregularities within the training 
areas, and the small mean separation among the signatures indicates 
many similarities among the data values of resolution elements in all 
training areas. Thus, mis classifications of those elements by the 
resulting signature set will be high. As resolution is degraded, the 
variance of each signature becomes smaller while the means for the 
most part remain unchanged, causing the amount of statistical overlap 
(competition) among the signatures to decrease. (Slight movement of 
some means is evident and is attributed to the random noise that was 
reinserted into the spatially degraded data.) Thus, resolution ele- 
ments in coarser resolution data should have higher probabilities of 
being correctly classified. The results of Ref. 1 indicated that 
higher classification accuracies were indeed achieved when averaged 
over all features and for some individual features. 

The fact that classification accuracies for some features show a 
different dependence on spatial resolution seems to be dependent on 
the amount of competition existing between any one feature signature 
and the signatures of other features. Figures 3-6 illustrate how the 
amount of overlap decreases dramatically for some signatures and only 
slightly, if at all, for others. Although all signatures represent 
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pine feature s 5 both a greater amount of illumated background mate- 
rials seen among the small trees within the Pine Regeneration feature 
and the somewhat larger proportion of shadows cast by the large conifer 
(and occasional hardwood) tree crowns in the Mature Shorcleaf Pine 
feature serve to displace the respective signature means for these 
features away from the centroid of all the signatures . Thus the capa- 
bility for accurate feature discrimination will be higher for these 
signatures by virtue of encompassing some amount of unique decision 
space. With decreasing variance for all signatures, feature discrimi- 
nation for these signatures improves, since overlap with neighbor- 
ing signatures decreases , thus enlarging their amount of unique deci- 
sion space. Figure 9 illustrates, for Mature Shortleaf Pine, the pro- 
gressive decrease in the misclassification of its resolution elements 
as spatial resolution was degraded. 

However, physical appearances for Immature Shortleaf Pine and 
Immature Loblolly Pine are less differentiable, and the increased 
proportion of hardwood tree crowns within the Mature Loblolly Pine 
feature results in a larger variance for this signature. The near 
co-location of these signature means with the centroid for all the 
signatures results in considerable overlap between each Of these 
signatures and its neighbors that, continues to exist despite a pro- 
gressive decrease in variance. Note that the signature for Immature 
Loblolly Pine seems to have an advantage at feature discrimination 
(see Figure 1) due to its relatively smaller variance, while the larger 
variance for the Mature Loblolly Pine signature detracts greatly from 
its feature discrimination capability. Figure 10 illustrates a consid- 
erable amount of misclassification for resolution elements within the 
Mature Loblolly Pine feature at all cases of spatial resolution. In 

Figure 6, the removal of the Immature Loblolly Pine signature at 
2 

(64 meters) serves to reduce much of the competition for Immature 
Loblolly Pine, thus, apparently causing the dramatic improvement in the 
classification accuracy illustrated for these two features in Figure lV 
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FIGURE 9. PERCENT OF RESOLUTION ELEMENTS WITHIN THE MATURE SH0RTLEAF 
PINE FEATURE THAT WERE MISCLASSIFIED AS EACH OF THE REMAINING 
FEATURES IN DATA SEGMENT 1 



FIGURE 10. PERCENT OF RESOLUTION ELEMENTS WITHIN '"HE MATURE LOBLOLLY 
PINE FEATURE THAT WERE MISCLASSIFIED AS EACH OF THE REMAINING 
FEATURES IN DATA SEGMENT 2 
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The fact that classification performance does generally increase 
for all features in Data Segment 2 is apparently due to the greater 
mea, s'-oaration among the feature signatures (Figures 7 and 8) a 
result of the more readily differentiable feature appearances. It is 
therefore likely that overlap among the resultant signature dis- 
tributions was less severe, thus allowing each distribution to encom- 
pass some amount of unique decision space. Decreases in variance 
caused by degrading spatial resolution would act to reduce overlap 
and further enlarge the unique decision space for each signature. 

Note that the use of two signatures to classify the Pine Regen- 
eration Feature in Data Segment 1 and three signatures to classify 
Immature Laurel Oak/Willow Oak Saw timber in Data Segment 2 likely 
influenced their relatively high classification performances demon- 
strated in Figures 1 and 2. Such use of multiple signatures to account 
for obvious nonuniform areas within the feature resulted in signature 
distributions having smaller variances than would have occurred for a 
single signature extracted from the same training area as for the 
multiple signatures. Thus, for regions of statistical overlap with 
neighboring signatures of larger variance, these signatures displayed 
an improved ability for classifying their respective features since 
misclassif ications of respect-ive resolution elements were lower, 

2.3.2 Simulation and Analysis of Signature Statistics 

The classification results displayed in Figures 1 and 2 were obtained 
with a standard single— element classification algorithm that utilizes 
signatures having multivariate normal statistics. To the extent that 
the data actually were normally distributed, one should be able to match 
the actual classification performance with a similar classification of 
simulated data generated from the signature statistics under an assump- 
tion of multivariate normality. The simulation results, thus, represent 
an ideal performance level to be expected under the normality assumption. 
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Departures of real results from the simulated results then would be 
attributable to non-normal distributions of data values within the 
feature training areas. 

Results obtained for the ideal expected classification performance 
of each data segmert generally displayed great similarities to the 
real classification performances illustrated in Figures 1 and 2. Figure 
11 illustrates the ideal expected classification performance for features 
of Data Segment 1 at each case of spatial resolution. Figure 12 shows 
the difference between the ideal expected performance and real performance 
expressed as the root mean square (RMS) error in percent accuracy between 
each feature's real classification performance and its ideal expected 
classification performance over all features in each data segment. The 
existence of error indicates some deviation from true normal distributions 
does exist for the data values within the training areas . The general 
decrease in error as spatial resolution degrades suggests that the data 
values within the training areas tend to approach a normal distribution 
more closely as the ground area of each resolution element increases. 

This is not surprising since an increased amount of ground area within 
a resolution element would tend to average out variations caused by the 
spectral irregularities within a feature area and reduce the likelihood 
of different modes that might occur within distributions of fine 
resolution data. 

To illustrate the effect of degrading spatial resolution on signature 
variance, we have plotted the values of determinants for signatures from 
Data Segment 1 in Figure 13. Each determinant represents the product of 
the signature eigenvalues and thus, is proportional to the square of the 
multi-dimensional dispersion volume of the signature distribution. A 
large decrease in determinant value indicates a large reduction in 
M signature size 1 '. 

It is interesting to note that the determinants for various signa- 
tures do not necessarily decrease at the same rate, as resolution degrades. 
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FIGURE 11. IDEAL GLASSIFICATION ACCURACIES TO BE EXPECTED FOR DATA SEG- 
MENT 1 ASSUMING THAT THE DATA VALUES WITHIN FEATURE TRAINING 
AREAS HAD MULTIVARIATE NORMAL DISTRIBUTIONS AT EACH 
CASE OF SPATIAL RESOLUTION 
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FIGURE 12. RMS DIFFERENCE OVER ALL FEATURES IN PERCENT ACCURACY BETWEEN 
THE REAL AND IDEAL EXPECTED CLASSIFICATION PERFORMANCE 
FOR EACH DATA SEGMENT 


2 ™ 


FORMERLY WALLOW RUN LABORATORIES. THE UNIVERSITY OF MICHIGAN 


Ct> 

rH 

td 

> 


JJ 

3 

5 

e 

ai 

jj 


aj 

a 



FIGURE 13. DETERMINANTS FOR SIGNATURES OF DATA SEGMENT 1 AS A FUNCTION 

OF SPATIAL RESOLUTION 
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For this set of signatures, the decrease in absolute magnitude for anv 
signature determinant is very large for each succesive change in resolu- 
tion, but differences in the rates of decrease among signatures, although 
small, are evident, We suggest that such non-uniform rates of change in 
determinant value may be indicative of some textural attribute that 
varies with spatial resolution. 

A further but inconclusive observation is provided by Figure 14, 
where, for each of two feature signatures, the square root of the de- 
terminant is compared to the product of the 11 channel standard deviations. 
This latter quantity is proportional to the multi-dimensional volume 
that would be occupied by a totally uncorrelated multivariate signature 
distribution. The actual hypervolume occupied by a distribution 
(proportional to the square root of the determinant) is less by virtue 
of the correlation that exists between channels. As spatial resolution 
degrades, the square root of the determinant decreases less than the 
product of the standard deviations, and therefore approaches that 
quantity. This indicates that the correlations between channels 
decrease as resolution is degraded. 
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FIGURE 14. COMPARISON OF SIGNATURE HYPERVOLUMES WITH THE RESPECTIVE 
PRODUCTS OF THE 11 CHANNEL STANDARD DEVIATIONS 
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3 

INVESTIGATION OF DECISION THRESHOLD EFFECTS 

3.1 BACKGROUND 

Figures 15 and 16 show the percent of unclassified resolution 
elements as a function of spatial resolution that were previously 
reported [l] for Data Segment 1 and Data Segment 2, respectively. These 
results had been obtained by using a standard single-element classifica- 
tion algorithm that incorporated a constant decision rejection threshold 
for all cases of spatial resolution. Figure 15 indicates that the per- 
cent of unclassified elements in Data Segment 1 is fairly constant except 

9 9 

when resolution size is (32 meters) “ and (64 meters) For Data Segment 

2 

2, only the (64 meters) case of resolution has a level of unclassified 
elements which is inconsistent with the other resolutions. 

3.2 PROCEDURE 

To investigate the effect of the classification rejection threshold 
for influencing the amount of unclassified resolution elements, we 
reclassified the data for the three cases of spatial resolution (as in- 
dicated above) having largi proportions of unclassified elements. Each 
data set was reclassified using no rejection threshold so that each 
element was forced to be classified as one of the forest features. The 
exponent of the multivariate normal density associated with the classi- 
fication of each element was preserved, and the classification results 
were tabulated for the exponent level which yielded approximately two 
percent unclassified elements for Data Segment 1 on boundary inclusive 
test sets. For Data Segment 2, the exponent level was adjusted to yield 
4.5 percent unclassified elements matching the other resolutions for 
that segment. 

3 . 3 RESULTS 

Figures 17 and 18 show the percent of unclassified elements as a 
function of spatial resolution, using threshold-adjusted classification 
results. Since the threshold was adjusted to provide a constant 
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FIGURE 15. PERCENT OF ELEMENTS IN DATA SEGMENT 1 WHICH ARE UNCLASSIFIED 
WITH THE THRESHOLD CONSTANT AS SPATIAL RESOLUTION VARIES 
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FIGURE 16, PERCENT OF ELEMENTS IN DATA SEGMENT 2 WHICH ARE UNCLASSIFIED 
WITH THE THRESHOLD CONSTANT AS SPATIAL RESOLUTION VARIES 
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FIGURE 17. PERCENT OF ELEMENTS IN DATA SEGMENT 1 WHICH ARE UNCLASSIFIED 
AS SPATIAL RESOLUTION VARIES WHEN THE THRESHOLDS ARE ADJUSTED 
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FIGURE 18, PERCENT OF ELEMENTS IN DATA SEGMENT 2 WHICH ARE UNCLASSIFIED 
AS SPATIAL RESOLUTION VARIES WHEN THE THRESHOLDS ARE ADJUSTED 
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proportion of unclassified elements in boundary inclusive test sets as 
spatial resolution varied, these test sets show little variance as a 
function of spatial resolution. Training sets and boundary exclusive 
test sets, which were classified using the threshold determined for 
boundary inclusive test sets, also now exhibit fairly consistent propor- 
tions of unclassified elements for all spatial resolutions. 

The previously used threshold level (chi square value), which gives 
a 0,001 level of significance for 11 channel data, was 31. for Data Seg- 
ment 1 levels for the threshold adjusted classifications were raised to 

2 2 2 
52 for (32 meters) and 128 for (64 meters) . The (64 meters) Data Seg- 
ment 2 classification required a threshold level increase :o 55 to give a 
proportion of unclassified elements consistent with the other cases of 
spatial resolution. 

Results are reported comparing the threshold-adjusted classifications 
to previous classifications using the 31 threshold level. The effects 
of these results on both classification accuracy and area proportion 
estimation are discussed. 

3,3.1 INFLUENCE ON FEATURE AND HIERARCHY CLASSIFICATION ACCURACY 

The comparison of classification results using different threshold 
2 2 

levels for (32) and (64 meters) spatial resolutions of Data Segment 1 
are given in Tables 1-3 for training sets, boundary exclusive test sets 
and boundary inclusive test sets, respectively. The percent correct 
classification is reported for each of the forest features in the 
respective hierarchies and the overall percent correct classification 
accuracy, based on total number of points correctly classified in the 
data segment, has also been calculated for each hierarchy, 

Training set classification accuracies (Table 1) remain the same 
or improve only slightly with the higher threshold levels for both cases 
of spatial resolution. This slight change is not surprising since 
Figure 15 shows that the percent of unclassified elements for training 
sets was small when the previous threshold levels were used. 
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TABLE 1, COMPARISON OF PERCENT CORRECT CLASSIFICATION FOR (32)2 AND 
(64 METERS) 2 TRAINING SETS IN DATA SEGMENT 1 USING CONSTANT 
THRESHOLDS VERSUS ADJUSTED THRESHOLDS 


(32M) 2 (64M) 2 


Constant 

Adjusted 

Cons tant 

Adjusted 

Threshold 

Threshold 

Threshold 

Threshold 

(31) 

(52) 

(31) 

(128), 


HlERARCUi: CONDITION CLASS 





CONIFER REGEN. (2.3) 

70.5 

70,5 

87.3 

90.1 

LOBLOLLY - IMM. (2.5) 

65.2 

65.2 

— 

■ — — 

LOBLOLLY - MATURE (2,6) 

19.6 

19.6 

44.4 

44.4 

SHORTLEAF - IMM. (1.3) 

35.6 

35.6 

71.4 

71.4 

SHORTLEAF - MATURE (1.4) 

75.6 

75.6 

85.0 

85.0 

OVERALL 

57.4 

57.4 

76.9 

78.0 


HIERARCHY : GROWTH STAGE 


CONIFER REGEN. (2.3) 

70.5 

70.5 

87.3 

90.1 

IMM. SAWTIMBER 

60.6 

61 . 9 

65.1 

65.1 

MATURE SAWTIMBER 

62.2 

62.2 

71.6 

73.1 

OVERALL 

65.1 

65.4 

76.2 

77.9 


HIERARCHY: COVER TYPE 


CONIFER REGEN. (2,3) 

70.5 

70.5 

87.3 

90.1 

SHORTLEAF PINE 

75.0 

75.0 

90.7 

90.7 

LOBLOLLY PINE 

44.8 

44.8 

45.7 

45.7 

OVERALL 

67*8 

67.8 

80.7 

81.8 


HIERARCHY: PHYSIOGNOMY 


CONIFER REGEN. (2.3) 

70 .5 

70.5 

87.3 

90.1 

PINE SAWTIMBER 

88.0 

88.5 

90.9 

91.8 

OVERALL 

81.2 

81.5 

89.5 

91.2 


ft 2 

The (64 meters): data set did not contain a signature far Immature 

Loblolly Pine (2.5) . 
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Comparison of Figure 11 with Figure 15 indicates that the more 

dramatic decreases in unclassified elements occurred when the higher 

threshold levels were used with the test sets, especially for the 
2 

(64 meters) case. Test set classification results (Tables 2 and 3) 

2 

show slightly improved overall accuracies for the (32 meters) case of 

2 

spatial resolution, and the (64 meters) case displays even more dramatic 

increases in the overall accuracies, ranging from 7.8 percentage points 

for condition class in boundary exclusive test sets to 13,0 percentage 

points for physiognomy in the boundary inclusive test sets. 

Table 4 compares the classification accuracies for individual 

forest features as well as the calculated overall accuracy of training 

sets and test sets in Data Segment 2, using the two different threshold 

2 

levels for the (64 meters) case. Training set overall classification 
accuracies improve by approximately four percentage points, but again the 
test sets show greater increases, i.e., 9 to 10 percentage paints, in 
overall accuracies. 

The larger increases in test sets can be explained by a comparison 
of Figures 16 and 18. Figure 16 shows that test sets contained a larger 
proportion of unclassified elements than training sets in the original 
classification. Thus, when the higher threshold level was used (Figure 18), 
test sets show a greater decrease in the proportion of unclassified 
elements than training sets. 

Overall classification accuracies for each hierarchy of features in 
Data Segments 1 and 2 are plotted as a function of spatial resolution 
in Figures 19-24. Each figure contains four plots. Plot (a) compares 
the overall accuracy for training, boundary exclusive test, and boundary 
inclusive test sets when the proportion of unclassified elements remains 
constant for all cases of spatial resolution. The other three plots, 
(b)-(d), show for each type of data set the differences between these 
classification accuracies and those obtained using a constant threshold 
level. 
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TABLE 2. COMPARISON OF PERCENT CORRECT CLASSIFICATION FOR (32) 2 AND 
(64 METERS) 2 BOUNDARY EXCLUSIVE TEST SETS IN DATA SEGMENT 1 


USING CONSTANT THRESHOLDS VERSUS ADJUSTED 

(32M) 2 

THRESHOLDS 

(64M) 

2'* 

Constant 

Adj us ted 

Constant 

Adjusted 

Threshold 

Threshold 

Threshold 

Threshold 

(31) 

(52) 

.(31). 

(128) 


HIERARCHY: CONDITION CLASS 


CONIFER REGEN. (2.3) 

71.4 

73.2 

78.6 

90.5 

LOBLOLLY - 

IMM. (2.5) 

56.8 

56.8 

— 

— 

loblolly - 

MATURE (2.6) 

19 .4 

19.9 

31.0 

38,1 

SHORTLEAF 

- IMM. (1.3) 

31.6 

31.8 

39-5 

44.4 

SHORTLEAF 

- MATURE (1.4) 

73.0 

73.0 

82.1 

82.1 

OVERALL 


52.2 

53.0 

59.9 

67.7 


HIERARCHY: GROWTH STAGE 





CONIFER REGEN. (2.3) 

71.4 

73.2 

78.6 

90.5 

IMM. SAWTIMBER 

53.5 

55.8 

40-2 

44.8 

MATURE SAWTIMBER 

54.2 

54.7 

59.3 

65.4 

OVERALL 

60.7 

62.3 

61.9 

70.1 


HIERARCHY: COVER TYPE 


CONIFER REGEN. (2.3) 

71.4 

73.2 

78.6 

90.5 

SHORTLEAF PINE 

67.0 

67.9 

80.0 

84,2 

LOBLOLLY PINE 

42.4 

44.4 

33.3 

39.6 

OVERALL 

64.6 

66 .0 

71.8 

79.6 


HIERARCHY? PHYSIOGNOMY 


CONIFER REGEN. (2.3) 

71.4 

73.2 

78.6 

90.5 

PINE SAWTIMBER 

82,8 

85.3 

80.4 

88.7 

OVERALL 

78.3 

80.6 

79.6 

89.5 


* 2 

The (64 maters) data set did not contain a signature for Immature 


Lob lolly Pine ( 2 . 5 ) . 
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TABLE 3. COMPARISON OF PERCENT CORRECT CLASSIFICATION FOR (32) 2 AND 
(64 METERS) 2 BOUNDARY INCLUSIVE TEST SETS IN DATA SEGMENT 1 
USING CONSTANT THRESHOLDS VERSUS ADJUSTED THRESHOLDS 


(32M) 2 ■ (64M) ^ 


Constant 

Adjusted 

Constant 

Adjusted 

Threshold 

Threshold 

Threshold 

Threshold 

(31) 

(52) 

(31) 

(128) 


HIERARCHY; CONDITION CLASS 
CONIFER REGEN. (2.3) 

70.1 

72.2 

76.3 

91.3 

LOBLOLLY - IMM. (2.5) 

47.1 

48.5 

— 


LOBLOLLY - MATURE (2.6) 

17.4 

17.8 

34.6 

40.4 

SHORTLEAF - IMM. (1,3) 

28.9 

29.1 

29.3 

35.3 

S110RTLEAF - MATURE (1.4) 

73.7 

74.1 

74.1 

79.3 

OVERALL 

50.1 

51.2 

54.3 

63.5 


HIERARCHY; GROWTH STAGE 


CONIFER REGEN . (2.3) 

70.1 

72.2 

76.3 

91.3 

IMM. SAWTIMBER 

48.5 

50.6 

28.5 

34.6 

MATURE SAWTIMBER 

53.8 

54.4 

58.2 

65.5 

OVERALL 

58.2 

59.9 

55.8 

65.8 


HIERARCHY; COVER TYPE 


CONIFER REGEN. (2.3) 

70.1 

72 .2 

76.3 

91.3 

SHORTLEAF PINE 

67.8 

70.1 

70.7 

80.5 

LOBLOLLY PINE 

40.3 

42.6 

37.9 

42.4 

OVERALL 

63.7 

66.0 

67.5 

78.5 


HIERARCHY ; PHYSIOGNOMY 


CONIFER IlEGEN . (2.3) 

70.1 

72.2 

76.3 

91.3 

PINE SAWTIMBER 

81.3 

84.9 

72,1 

83.8 

OVERALL 

77.1 

80.1 

73.8 

86.8 

* ; 9 ■ 

The (64 meters) data set 

did not 

contain a 

signature for 

Immature 


Loblolly Pine (2.5) . 
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TABLE 4. COMPARISON OF PERCENT CORRECT GLASSIFICATION FOR (32) 2 AND &6A 'METERS ) 2 CASES IN 
DATA SEGMENT 2 USING CONSTANT THRESHOLDS VERSUS ADJUSTED THRESHOLDS 


TRAINING 
SETS ;; 


Constant Adjusted 
Threshold . Threshold 


BOUNDARY EXCLUSIVE 
.! TEST SETS 


Constant 
Threshold 

(31) V (55) 


Adjusted 

Threshold 


BOUNDARY INCLUSIVE 
T EST SETS 

Constant Adjusted 

Threshold Threshold 
(31) (55) 


HIERARCHY : CONDITION CLASS 

tOBLOlLY - JAM-. : (2, 5) > :: 

LAdREL” OAK/WILl OW OAlt (3 .1) 
SWT ONM/td.- OAK / VI i CM (4 . 2) 
CUT OVL^(7.1) : ; 'I 

; ' GVERALK\1* : 

■ OVERALL V s * 


HIERARCHY,; PHYSIOGN OMY - 
;v “■ 


100,0 

100.0 

■ 86 . O', 

90.0 

V 72.9 , 

80.0 

86.8 : 

92.6 

62.i \ : 

72.4 

l 59.7 

68.4 

48.1 

49.4 

50.0 \ 

50.0 

: : 50.0 

50.0 

95.8 

100.0 

75.4 \ 

: \ 

90.2 

74.2 

■: 87.6 

76.3 

79.1 

69.3 \ 

79.2 

64,9 

73.9 

92.6 

96.3 

V 
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Overall calculated over all sets. , 

Overall calculated omitting 4.2 training set. 


QONIFER SAUTIMBER (2., 5) S' 

100.0 

100.0 

86.0 

90.0 

72.9 

80.0 

D 

3 

HARDWOOD SAWTIMBER 

89.8 

95.2 

84.3 

94.1 

85.5 

94.5 

R 

£A 

CUT OVER (7.1) 

95.8 

100.0 

75.4 

90 . 2 

74.2 

87.6 

H 

■ -2 

m 

c 

OVERALL 

92.6 

96.7 

82.6 

. 92.4 ■ ' 
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' 90.2 
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(d) Changes For Boundary Inclusive Test Sets 


FIGURE 19, EFFECTS OF ADJUSTED DECISION THRESHOLDS ON CLASSIFICATION ACCURACY AS A FUNCTION OF 

SPATIAL RESOLUTION FOR CONDITION CLASSES OF DATA SEGMENT 1 
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(a) Revised Results For All Data Sets 
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(c) Changes For Boundary E^clusivt. Test Sets 


FIGURE 20. EFFECTS OF ADJUSTED DECISION THRESHOLDS 

SPATIAL RESOLUTION FOR GROWTH 
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(b) Changes for Training Sets 
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(c) Changes For Boundary Exclusive Test Sets 



(d) Changes For Boundary Inclusive Test Sets 


FIGURE 21. EFFECTS OF ADJUSTED DECISION THRESHOLDS ON CLASSIFICATION ACCURACY AS A FUNCTION OF 

SPATIAL RESOLUTION FOR COVER TYPES OF DATA SEGMENT 1 
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(a) Revised Results For All Data Sets 






(d) Changes For Boundary Inclusive Test Sets 


FIGURE 22, EFFECTS OF ADJUSTED DECISION THRESHOLDS ON CLASSIFICATION ACCURACY AS A FUNCTION OF 

SPATIAL RESOLUTION FOR PHYSIOGNOMY IN DATA SEGMENT 1 
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FIGURE 23. EFFECTS OF ADJUSTED DECISION THRESHOLDS ON CLASSIFICATION ACCURACY AS A FUNCTION OF 

SPATIAL RESOLUTION FOR CONDITION CLASSES OF DATA SEGMENT 2 
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(c) Changes For Boundary Exclusive Test Set 
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(d) Changes For Boundary Inclusive Test Sets 


FIGURE 24. EFFECTS OF THE ADJUSTED DECISION THRESHOLDS ON CLASSIFICATION ACCURACY AS A FUNCTION 

OF SPATIAL RESOLUTION FOR PHYSIOGNOMY IN DATA SEGMENT 2 
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Figures 19-22 present similar plots for each of the four hierarchies 

of Data Segment 1. In every case for threshold adjusted results (Plot (a) 

2 

of Figures 19 through 22), the highest accuracies are for the (64 meters)" 
case. For training and test sets, there is a gradual increase in per- 
formance as spatial resolution coarsens with a larger increase at 
2 

(64 meters) . The plots which compare the threshold ad-justed results 

with the constant threshold results (Part (b)-(d) of Figures 19-22) show 

2 

a significant improvement only for (64 meters) test sets. These 
increases in classification accuracy change the trend of boundarv in- 
clusive test sets for growth stages and physiognomy (Figures 20(d) 

2 

and 22(d)) which previously Indicated that the (32 meters) resolution 
gave the highest accuracy. 

Figures 23 and 24 are plots of the results for the two hierarchies 

of Data Segment 2. Plot (a) of each figure indicates that in most cases, 

2 2 
the (64 meters) resolution Is slightly better than the (32 meters) , 

but the amount of improvement in classification accuracy decreases after 
2 

(32 meters) for all cases except training sets of the condition class 
hierarchy. In contrast, the previous results (Plots (b)-(d) of Figures 23 

and 24) indicated a very significant drop in accuracy for the (64 me- 

2 2 
ters) case as compared to (32 meters) for test sets. 

One explanation of the slight increase in accuracy for the 
2 

(64 meters) case of Data Segment 2 is that, for physiognomic 
training sets (Figure 24(a), (32 meters)" 4, accuracy approaches 
100 percent. For all resolutions, the overall accuracies of both 
training and test sets are always higher In Data Segment 2 than in 
Data Segment 1. These higher accuracies reflect the fact that a 
classification of Data Segment 2 is trying to separate forest fearures 
which are more readily distinguishable than those in Data Segment 1 

and represents a different problem. The small difference in accuracy 

2 9 

seen for the (64 meters) case compared to the (32 meters) case in 

Data Segment 2, thus, may represent a plateau for cla isification of that 
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data which is apparent for the training set case for physiognomy 
(Figure 24(a)). Data Segment 1 apparently has not reached the limiting 
spatial resolution for classification accuracy of that data set. 

3.3.2 INFLUENCE ON AREA PROPORTION ESTIMATION 

The proportion of unclassified elements greatly affected the esti- 
mation of the proportion of each forest feature present in the area. 
Since the unclassified elements could not be included in the proportions 
of any of the forest features, estimates were lowered producing situa- 
tions in which every forest feature was underestimated. 

In Figures 25 and 26, RMS error is plotted as a function of spatial 
resolution for the physiognomy hierarchies of both data segments, 

RMS errors for each data segment were calculated as 



where: p_^ = ground truth proportion for one feature in the 

test area, 

p_^ = estimated proportion for the same feature in the 
test area, 

N = number of features considered. 

Figure 25 gives RMS error calculated from proportions of all elements in 
the test area while Figure 26 gives RMS error calculated from proportions 
using only classified elements. There is not a significant change in 
the results displayed in Figure 26 for threshold adiusted versus threshold 
constant results. However, Figure 25 shows a marked decrease in the 

2 t 

threshold-adjusted RMS error at (64 meters) for both data segments. 

Thus, when a large percentage of the elements in an area were unclassified 
and these elements were included in proportion calculations, the RMS 
error was significantly higher. 
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FIGURE 25, RMS ERROR, CALCULATED FOR ALL ELEMENTS IN EACH DATA SEGMENT, 
PLOTTED AS A FUNCTION OF RESOLUTION 
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FIGURE 26. RMS ERROR, CALCULATED FOR ONLY CLASSIFIED ELEMENTS IN EACH 
DATA SEGMENT, PLOTTED AS A FUNCTION OF SPATIAL RESOLUTION 
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Figures 27-30 are plots of the difference in percentage points 
of the forest feature proportions as determined by ground truth and 
estimated proportions of features. In Figures 27 and 28, the estimated 
proportions were calculated from all elements in the test area, but 
Figures 29 and 30 give results using only classified elements. The 
major difference in using the threshold adjusted results instead of the 
constant threshold results is that even at coarse spatial resolutions 
the results for all elements is very similar to the results for only 
classified elements. The threshold adjusted proportions seem to be 
a much more valid assessment of the accuracy attainable with proportion 
estimation since we no longer have the situation where everything is 
underestimated as for the constant threshold results in Figure 28, 
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FIGURE 27. PERCENT DIFFERENCE BETWEEN TRUE GROUND PROPORTIONS AND ESTIMA- 
TED PROPORTIONS CALCULATED FOR ALL ELEMENTS OF DATA SEGMENT 1 
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FIGURE 28. PERCENT DIFFERENCE BETWEEN TRUE GROUND PROPORTIONS AND ESTIMA- 
TED PROPORTIONS CALCULATED FOR ALL ELEMENTS OF DATA SEGMENT 2 
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FIGURE 29. PERCENT DIFFERENCE BETWEEN TRUE GROUND PROPORTIONS AND ESTIMA- 
TED PROPORTIONS CALCULATED FOR CLASSIFIED ELEMENTS OF DATA SEGMENT 1 
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FIGURE 30. PERCENT DIFFERENCE BETWEEN TRUE GROUND PROPORTIONS AND ESTIMA- 
TED PROPORTIONS CALCULATED FOR CLASSIFIED ELEMENTS OF DATA SEGMENT 2 
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4 

CONCLUSIONS 


In order to explain large variations in classification accuracy 
that were manifested by individual condition class forest features in 
processed multispeetral scanner data and the inconsistent changes in 
accuracy among the features that were noted as spatial resolution was 
degraded, wo generated two-channel ellipse plots which illustrated the 
forest feature signatures aL several cases of resolution. Analysis of 
the plots show that the capability of signatures to identify their 
respective features is dependent on the relationship of each signature 
to all others in the signature set. Signatures for which considerable 
statistical overlap or competition exists with neighboring signatures 
will produce low classification performance for their respective features 
due to large amounts of resolution element misclassif icstion. In such 
situations, signatures with small variance and high correlation (tighter 
distributions) may have an advantage over signatures with large variance 
and/or lower correlation. The use of multiple signatures to characterize 
obviously nonuniform areas within a feature is one method for producing 
signatures of smaller variance that would improve feature classification 
performance. 

Reductions in signature variance that occur in data of degraded 
spatial resolution cannot be said to improve classification performance 
for all forest, features a priori .. Signatures for forest features having 
large variances and small mean separations may continue to have much 
overlapping variance in coarser resolution data, thus causing little 
or no change in the total proportion of resolution elements misclassif led 
within their feature areas. 

A simulation of feature classification was performed under 
the assumption that the data values within feature training areas 
at each case of spatial resolution had multivariate normal distributions. 

A comparison of the "expected" performance to the real data classification 
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performance suggests that the >:eal data values within training /areas 
tended to approach, a normal distribution more closely as spatial resolution 
was degraded. This trend is attributed to the averaging of spectral 
irregularities wi t hin resolution : elements of pr ogres sively larger size 
and the consequent decreasing likelihood of different modes that might 

occur within distributions of fine resolution data. 

Small differences in rates of decrease for determinant values 
were seen to exist among the signatures of Data Segment 1 as spatial 
resolution was degraded . 1 t is suggested that such non-uniform rates . 

of change may be indicative of some textural attribute that varies 
with spatial resolution. In addition, evidence was provided to 
suggest that c r v: i olatiOns between the channels of a spectral signature 
decrease as spatial resolution is degraded . . 

In the second part of the study, adjustments of the threshold 
to maintain a constant proportion of unclassified elements among all 
cases of spatial resolution significantly increases- the overall, classifi- 
cation accuracy for (64 meters) “ data for both data Segments. As a re- 

2 ■■■■ ' 

suit of the increased a c .-uracies ' for (64 meters) ■ ■ data, the resolution •• 

which provides the most accurate over all elamant-by-element classifica- 

2 

tion results is (64 meters) - for threshold-adjusted results, The in- 
crease from (32 meters) ^ to (64 meters) is much more dramatic for 
Data Segment 1 than Data Segment 2. Accuracies are higher for Data 

Segment 2 than Data Segment 1 for all resolutions , "’thus the fact that 

2 • ; 2 rij "4 : 

the increases are relatively slight from (32) to (64. meters) may be 

2 

due to the already high (32 meters) accuracy. Classification results 
using adjusted thresholds, thus, indicate that the optimum spatial 
resolution for classification accuracy was (64 meters)"* not (32 meters) .. 

As a f dsait; of this study, we recommend for fu.tui^-ripexatidnal 
systems that decision thresholds to a constant 

value. For examp l^iir-c^Tig^study, where all ground features were repre- 
sented by training data, the area proportion .estimations should have been 
determined using non-threshclded classifications: which would ha^e forced 
the classification of all elements. 
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